
Attor^Mttocket No.: 066 1 8/8900 1/CIT-3 102 



APPLICATION 
FOR 

UNITED STATES LETTERS PATENT 



TITLE: PROGRAMMING SYSTEM AND THREAD 

SYNCHRONIZATION MECHANISMS FOR THE 
DEVELOPMENT OF SELECTIVELY SEQUENTIAL AND 
MULTITHREADED COMPUTER PROGRAMS 

APPLICANT: JOHN THORNLEY, K. MANI CHANDY AND HIROSHI 
ISHII 



CERTIFICATE OF MAILING BY EXPRESS MAIL 
Express Mail Label No. EL528179775US 



I hereby certify under 37 CFR §1.10 that this correspondence is being 
deposited with the UniteckStahts Postal Servicers Express Mail Post 
Office to Addressee wjjn sufficient postage on«ie\date indicated below 
and is addressed to thyAssistant f. ommissione/ for Patents, Washington, 
D.C. 20231. 



Date of Deposit I 






Signature / ^ 







Typed or Printed Natrte ofPerson Signing Certificate 



' Attorney's Dockj^^. 



06618/389001/CIT-3102 



PROGRAMMING SYSTEM AND THREAD SYNCHRONIZATION MECHANISMS FOR THE 

DEVELOPMENT OF SELECTIVELY 
SEQUENTIAL AND MULTITHREADED COMPUTER PROGRAMS 



The present application claims priority under 35 U. S.C. 
119(e) from provisional application number 60/112,817 filed 
December 17, 1998. 



Background 

Many computer programs are computationally intensive, 
meaning that they require large amounts of computing power. As 
a consequence, these programs may execute more slowly than the 
computer user desires, even on the fastest computers. One way 
of increasing the execution speed of a computationally intensive 
computer program is to divide the program into multiple units, 
or loci, of concurrent execution. . These units of execution are 
known as "threads" . A program with multiple threads of 
execution is known as a "multithreaded program" . A program with 
only a single thread of execution is known as a "sequential 
program" . The threads that make up a multithreaded program may 
be executed concurrently on multiple computer processors, 
allowing many operations in the program to be carried out 
simultaneously, thereby speeding up program execution. 
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In a multithreaded program, the program or operating 



system must control the access of threads to data objects in the 
program, in order to the prevent the multiple threads from 
concurrently accessing the same data object in an undesirable 

5 manner. If multiple threads modify the same data object 
concurrently, or read and modify the same data object 
concurrently, the resulting state of the program is extremely 
difficult to determine. Developing a multithreaded program is 
significantly more difficult than developing a sequential 

0 program because of the problems of (1) expressing the division 
of a program into multiple threads and (2) structuring and 
controlling the access of those threads to data objects. 



("Sthread") system with thread synchronization and production 
mechanisms . 



multithreadable code can be annotated using information 
0 indicative of its multithreadability . The multithreadable code 
constructs are code constructs that can be executed in a 
multithreadable manner, or equivalently in a sequential manner. 
Multithreadable code constructs may be expressed by annotating 
sequential code constructs to indicate that their multithreaded 



Summary 
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The present application teaches a structured thread 



Another aspect produces multithreadable code. The 
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execution is equivalent to sequential execution. The 
multithreadable code can be used along with multithreaded code. 
Specific instances of a multithreadable constructs: a 
multithreadable block, and a multithreadable for loop, and are 
5 disclosed. 

The second aspect of the system is the integration of 
multithreadable code constructs with traditional explicitly- 
multithreaded code constructs. Explicitly multithreaded code 
constructs must always be executed in a multithreaded manner. 
j: J0 Examples of explicitly multithreaded code constructs include 
,: S multithreaded block constructs, multithreaded for loop 

lx constructs, and library-based thread creation functions. 

t .. 

h £ 

rn Multithreadable code constructs and explicitly multithreaded 

5' ~~: 

:: code constructs may be intermingled within a program as 
l'Q.5 required, with well-defined meaning. 

! According to a first aspect , a special counter called an 
^ "s-counter", is used as a thread synchronization mechanism. 

Special "s-Flags" can also be used for thread synchronization, 
and flag synchronization is also described herein. 
20 Yet another aspect is the implementation of the programming 

system within an existing compiler environment using a special 
pre-processing system . 

The embodiments of the invention describe additional 
details , including the following : 
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The s-counter synchronizing the access of threads to shared 
data objects. The mechanisms use "monotonic" synchronization 
objects, with operations that can be constrained to only move 
the value of the object in one direction. Monotonic 
5 synchronization objects can be used to synchronize the access of 
threads to shared data objects in multithreadable code 
constructs in a manner that guarantees the equivalence of 
sequential and multithreaded execution. Specific instances of 
monotonic synchronization objects are disclosed, namely a form 
10 of counter called an "s-counter" and a form of flag called an 
; ^ "s-flag". The s-counter is a particularly powerful thread 
synchronization mechanism in many contexts, with its use in 

M 

j-.s multithreadable code constructs being one example. 
K " The application describes implementation of the 

l. A 

j ! |5 multithreaded programming system within an existing program 

development and compilation environment using a special source - 

'Q to-source preprocessing system and high-level thread library. 
This allows the system to be transparently and seamlessly 
integrated with existing programming systems such as Microsoft 

20 Visual Studio for the Microsoft Windows family of operating 

systems, the GNU program development tools for Linux and other 
versions of the Unix operating system, or on any version of the 
Java programming language, for example. 
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Brief Description Of The Drawings 
These and other aspects will now be described in detail 
with respect to the accompanying drawings, wherein: 

Figure 1 is a process flowchart showing a prior art method 
5 for compiling multithreaded code; 

Figure 2 shows a computer system and its thread allocation 
system; 

Figure 3 shows a flowchart of defining an s-counter; and 
Figure 4 shows a flowchart of Sthreads execution of a 
10 program. 

£ Detailed Description 

n FIG. 1 is a process flowchart showing a prior art method 

for compiling multithreaded code. Source code text 300 including 
15 multithreaded code constructs is processed by a conventional 
^ compiler 302. The compiler communicates with a linker 304 which 
^ links pre-existing routines from a library 306 with the output 
of the compiler to create an executable module 308. 

Existing operating systems, including the WIN32 API, often 
2 0 provide a general purpose thread library which may allow 

carrying out defined tasks like these. For example, a first 
thread may be defined for operating the CD ROM, and another for 
the modem. 
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Windows NT WIN3 2 thread creation is unstructured. A thread 
is created by passing a function pointer and an argument pointer 
to a CreateThread call. The new thread then executes the given 
function with the given argument. There is no specific 
5 relationship between the created thread and the creating thread: 
the two threads are effectively asynchronous. One thread for 
example, can arbitrarily suspend, resume or terminate the 
execution of another thread. 

This is not a problem for unrelated tasks like CD/modem 
^10 tasks noted above. However, when two parts of a program are to 
t: f be executed as threads, the synchronization operations are often 
r«* complex and error prone. Unpredictable interactions among the 
i-n multiple threads can induce problems including race conditions, 
i; and deadlock. Effectively, the user is left with the daunting 
|15 task of using these thread libraries in a way that does not 

ii zz 
: 

cause this problem. 
^ The present application discloses a specific embodiment 

operating using the Windows NT (TM) system. It should be 

understood, however, that this system is portable across many 
20 platforms and that the same concept described herein can be used 

in those systems, including Linux, and any other operating 

system. 

While a process has its own address space, a thread is 
often simply a program counter and stack pointer. A process may 
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have many threads but all the threads share the same address 
space . 

Figures 2 and 3 show this operation in a computer system. 
Figure 2 shows a computer system, with four processors 200 , 2 02, 
5 204, 206. The processors can be in a multiple processor system 
as shown. The pool 199 of processors is associated with an 
operating system 210, a user interface 215, auxiliary hardware 
220 (e.g. memory, chipsets, etc),a display 225 and other 
computer components. 
10 The operating system 210 includes multiple threads 212, 

■ : i 214, and others. Each thread is resident on the stack within 
;'V the heap. The threads are associated with processors, which 
«ft execute the threads . Figure 2 shows the pool of threads on the 
... left and the pool of processors on the right. Each of the 
|ll 5 dynamically-created threads are peers. Of course there can be 
«a many more threads than processors. The operating system 

J : 
*: k: 

" ; :3 controls the threads to dynamically switch between the 
processors . 

The present application defines an entirely new way of 
20 creating, synchronizing, and handling the synchronization among 
threads. The system uses a new way of compiling code based on 
multithreadable code, either alone, or in conjunction with 
multithreaded code. 
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The operating system or programming language has a higher 
level system that uses special constructs called "equivalency 
annotations" . A lower level function call based system is used 
with special objects. Those special objects can synchronize 
5 among the threads in a way that prevents the objects within the 
threads from having ambiguous states . 

Many of these systems are based on the concept of 
equivalence annotations. Equivalence annotations can take many 
forms - pragmas,, special keywords, special kinds of comments, 
10 special characters, textual modifications (such as boldfacing, 

underlining, or italics), and others. They could be part of the 
: ; ; program text, or in a separate file i.e., an extra file that 

):s contains nothing but the annotations. The pragma can have 

x, • ■ 

I: si 

meaning to a compiler. Pragmas often form a specified syntax, 

v ™ 

rJ5 but usually convey nonessential information that is intended to 
help the compiler to optimize the program. 

^2 The present embodiment uses these pragmas as special 

equivalence annotations. Pragmas are convenient for annotations 
since many programming language already provide pragmas for 

20 other purposes. While a pragma is described as being used as 

the preferred annotation of the present application, the program 
can certainly be annotated in other manners. For example, Java 
does not support pragmas, so a special kind of comment line 
could be used. The equivalence annotations described throughout 
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this specification should be understood to be interpretable in 
this way. 

The mult i threadable equivalence annotation can be a pragma when 
embodied in the C programming language. This indicates that a 
5 block or loop can be executed in a multithreaded manner. This 
means that there is no timing dependent nondeterminacy , and the 
system can execute the instructions into a multithreaded system. 

The mult i threaded equivalence annotation means that a block or 
loop must be executed in a multithreaded manner. The 
10 multithreaded execution need not be equivalent to sequential 

execution. Lock synchronization can be used to introduce 
!'"'.' nondeterminacy if desired. 

j:s The equivalence annotation becomes part of the operating 

s "" system. Special, monotonically increasing and otherwise 
i : |5 constrained s-counters, and similarly constrained s-flags are 

operated to synchronize the access of threads to shared memory 
U in order to prevent unwanted interference. 

A special synchronization counter, or s-counter is defined as an 
object with three basic attributes. The s-counter has a non- 
20 negative integer value. The object only allows an increment 

operation and a check operation. An initial value of the s-counter 
object is set to zero. An increment function automatically 
increases the value of the counter by a specified amount. The 
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check operation suspends the calling thread until the value of the 
counter becomes greater than or equal to a specified level. 

The multi-threaded programming system has a higher level 
notation includes annotation objects in the program code. Using 
the example of the c language, this can be described as "multi- 
threaded c" . 

A lower level structured thread library is described as 
"Sthreads" . The special annotation objects are transformed into 
special Sthread calls by the Sthreads annotation objects pre- 
processor. 

The multithreaded model uses the thread 
synchronization/annotation objects disclosed above to 
synchronize among the threads. 

Threads can be created in different ways. A first thread 
creation construct is the multithreaded block. This is indicated 
by the multithreaded keyword placed immediately before an ordinary 
C block: 

MULTITHREADED { 

statement 
statement 

} 

This notation specifies that the statements of the block 
should be executed as asynchronous threads. This is a 
conventional way of referring to these threads. For example, 
the operating system could create threads to read from CD, and 
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threads to read from tape . The threads are executed and proceed 
concurrently. They all share the same address space as the 
parent program. Execution does not continue past the 
multithreaded block until all the threads have individually 
terminated. It is typically illegal for the program to contain 
any kind of jump between the individual statements of the block, 
from inside the block to outside the block, or from outside the 
block to inside the block. 

A second thread creation construct is the multithreaded 
for- loop, indicated by the multithreaded keyword placed immediately 
before an ordinary for-loop: 

MULTITHREADED 

for (i = expression; i comparison expression; i = i + expression) 
statement 

This notation specifies that the iterations of the loop 
should be executed as asynchronous threads. The threads all 
share the same address space as the parent program. Each 
thread, however, has a local copy of the loop control -variable 
with a different value from the iteration range. The iteration 
scheme can restrict to a single control -variable and expressions 
that are not modified within the loop body. Execution does not 
continue past the multithreaded for-loop until all the threads 
have individually terminated. It is illegal for the program to 
contain any kind of jump from inside the loop to outside the 
loop or from outside the loop to inside the loop. In essence, a 
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multithreaded for- loop is a quantified form of multithreaded 
block. 

Multithreaded and ordinary blocks and for- loops can be 
arbitrarily nested. 

Traditional approaches have often been categorized as 
either being explicitly multithreaded or implicitly 
multithreaded. With explicit multithreading, the programmer 
expresses exactly how the operations of the program are executed 
by threads. Implicit multithreading is carried out when the 
programmer writes an ordinary sequential program. The 
programming system, e.g. the compiler, determines how the 
operations can be executed by separate threads. 

The present application goes beyond the multithreaded 
concepts described above into a concept of multithreadable code 
constructs. The multithreadable construct can be executed 
according to a specified sequential operational semantics. The 
most common operational semantics would be. executed 
sequentially. An alternative, however, allows the 
multithreadable code construct to be operated according to 
multithreaded operational semantics. 

Rules are defined that constrain the multithreaded 
execution such that its result is equivalent to sequential 
execution. 
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As disclosed herein, the multithreadable code construct is 
formed of : 

i) a syntactic description of the form of the construct, 

(ii) a sequential operational semantics, that, when 
executed, defines how to execute the construct sequentially, 

(iii) a multithreaded operational semantics, defining how 
to execute the construct by a set of threads, and 

(iv) a set of implicit or explicit programming rules that 
are sufficient to ensure the equivalence of sequential and 
multithreaded execution of the construct. 

The multithreadable pragma becomes an assertion by the 
programmer that the block or for loop can be executed in a 
multithreaded manner without changing the results of the 
program. The multithreadable pragma can be applied to blocks and for 
loops in which the statements or iterations are independent of 
each other. The multithreaded execution is equivalent to 
sequential execution in this case. It is not a directive that 
the block or for loop must be executed in a multithreaded manner. 

As a simple example, consider the following program to sum 
the elements of a two-dimensional array: 

void SumElements (float A [N] [N] , float *sum, int numThreads) 

{ 

int i ; 

float rowSum [N] ; 

#pragma multithreadable mapping (blocked (numThreads) ) 
for (i = 0; i < N; i++) { 
int j ; 
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rowSum[i] = 0.0; 

for (j = 0; j < N; j++) 

rowSum[i] = rowSumti] + A[i] [j] ; 

} 

5 *sum = 0.0; 

for (i = 0; i < N; i++) 

*sum = *sum + rowSum[i] ; 

} 

Multithreaded execution of the for loop is equivalent to 
10 sequential execution because the iterations all modify different 
rowSum[i] and j variables. The arguments following the pragma 
indicate that multithreaded execution should assign iterations 
to numThreads different threads using a blocked mapping. There is 
a rich set of options that control the mapping of iterations to 

V s ] 

if 5 threads . 

i;f1 Therefore, the Multithreaded C preprocessor has two modes: 

V ^ 

H sequential mode in which the multithreadable pragma is ignored, and 
^ multithreaded mode in which the multithreadable pragma is 
!~ transformed into Sthreads calls. Programs can be developed, 
l 20 tested, and debugged in sequential mode, then executed in 

multithreaded mode for performance. In addition, performance 
analysis and tuning can often be performed in sequential mode. 

Determinacy of results is an important consequence of the 
equivalence of multithreaded and sequential execution. If 
25 sequential execution is deterministic (which is usually the 
case) , multithreaded execution must also be deterministic. 
Determinacy is usually desirable, since program development and 
debugging can be difficult when different runs produce different 
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results. In other multithreaded programming systems, determinacy 
is difficult to ensure. For example ,. locks , semaphores, and 
many-to-one message passing almost always introduce race 
conditions and hence nondeterminacy . However, nondeterminacy is 
5 important for efficiency in some algorithms, e.g., branch-and- 
bound algorithms. 

Multithreaded and multithreadable code constructs are 
integrated in this system. The programming system incorporates 
both explicitly multithreaded constructs which must be executed 
10 according to the multithreaded semantics, along with 
: ^ multithreadable constructs which may selectively executed 
\; according to their sequential or multithreaded semantics. The 
s multithreaded constructs are generally used to express 

multithreaded algorithms that have no sequential equivalent. 
0.5 This can include controlling different hardware that haw no 
« integration with one another, or controlling simultaneous 
=j different windows in a graphical user interface. 

Multithreadable constructs are used to express the 
opportunity to use multithreading to speed up the execution of a 
20 computationally- intensive algorithm, by using multiple threads 
on multiple processors. 

By using both multithreaded and multithreadable constructs, 
the operating system can use one thread to control each window 
with multithreaded constructs, and the output to each window, 
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within a window, is computed faster with the multiple threads 
using multithreadable constructs. 

As described above, the synchronization can be carried out 
by an entry s-counter, or an s-flag. Each are defined to have 
5 certain constraints. 

An S-counter, defined in the context of the C programming 
language, is diagrammed in Figure 3. It can be defined as a 
type definition and a set of interface functions. The counters 
are encapsulated as a class in an object-oriented language such 
.10 as C++ or Java. The definition of the fundamental programming 

'.: si 

: = interface for S-counters is as follows: 

M typedef counter type definition Counter; 

j":= int InitializeCounter (Counter *c) ; 

*;15 /* Initializes value (c) to zero. */ 

\J /* Must be called only once, before all other operations on counter c. */ 

= , int FinalizeCounter (Counter *c) ; 

r '~ /* Must be called only once, after all other operations on c. */ 

S30 

int CheckCounter (Counter *c, unsigned int level); 
/* Suspends until value (c) greaterorequal level. */ 

i : 3 int IncrementCounter (Counter *c, unsigned int amount); 

:§5 /* Increases value (c) by amount. */ 

An s-counter object c implicitly has a nonnegative integer 
attribute value (c), which can only be accessed through the 
interface functions. The Initialize function at 300 initializes 
30 value (c) to zero or some initial value. 

Importantly, the counter is monotonic, as illustrated in 
310. No decrement function is defined. Its value monotonically 
increases . 
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Any attempt to check the counter, shown as step 32 0, 
suspends the calling thread at 325. This prevents a condition 
which can catch or miss some action occurring during the check 
operation. Each s-counter maintains a dynamic list of thread 
suspension queues 330, with one queue for each value on which at 
least one Check operation is suspended. 

Check compares value (c) to level and suspends until value (c) 
becomes greater than or equal to level. This is generically 
shown as AWAKE in step 340. Increment at 310 atomically 
increases value (c) by amount, thereby reawakening all Check 
operations suspended on values less than or equal to the new 
value (c) . 

All the functions can return an error code. Possible error 
conditions include invalid arguments, operations on an 
uninitialized counter, and counter overflow. 

The type definition described above is carefully selected 
to remove the possibility of race conditions occurring on 
counter synchronization. There is no Decrease operation. 
Therefore, the value of an s-counter is monotonically increasing. 
There is no possibility of a Check operation missing an Increment 
operation since check suspends the thread. There is no Probe or 
nonblocking Check operation. It is recognized by the inventor 
that any instantaneous value may depend on the relative timing 
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of the individual threads. Therefore, no operation can be based 
on the instantaneous value of an s-counter. 

A Reset operation can also be used to efficiently reuse 
counters between different phases of a program. 

Alternatively, the old counters can be deleted and 
recreated as new counters. Reset simply resets value (c) back to 
zero. However, to avoid the possibility of race conditions, Reset 
must not be called concurrently with any other operation on the 
same counter. Reset ends the process, and is not intended as a 
means of synchronization between threads . 

Another thread synchronization object is a special flag, 
called an s-flag. S-Flags, like s-counters, have restricted allowed 
operations within the multithreadable code concept. S-Flags 
support Set and Check operations. Initially, an s-flag is not set. 
A Set operation on an s-flag atomically sets the flag. A Check 
operation on a flag suspends until the flag is set. Once an s-flag 
is set, it remains set. 

Flags and counters are provided to provide deterministic 
synchronization within multithreadable constructs, as previously 
described . 

In summary of the above, an s-counter object has the 
following operations (expressed in the C programming language) : 

Initialize (Counter *c) 



- 18 - 



Attorney's et No. 06618/389001/CIT- 

Finalize (Counter *c) 

Increment (Counter *c, unsigned int amount) 
Check (Counter *c, unsigned int value) 
Reset (Counter *c) 

5 

The Initialize operation initializes the Counter object and 
sets its value to zero. The Finalize operation destroys the 
Counter object. An Increment operation increases the value of 
the Counter object by amount. A Check operation suspends the 

10 calling thread until the value of the Counter object is at least 
value. A Reset operation resets the value of the Counter object 

S to zero. 

;pj In the following simple example, a "producer thread" 

produces items and writes them to a buffer. A group of one or 

35 more concurrently executed "consumer threads" each independently 
reads the items from the buffer. The key synchronization issue 
is to prevent the consumer threads from reading items from the 

; !i buffer that have not yet been written by the producer thread. 
The following program fragment gives implementations of the 

20 producer thread and consumer threads (in the C programming 
language) using a counter for synchronization . 



Counter count; 

Item buffer [NUM_ITEMS] ; 

25 

ProducerThread (int blockSize) 
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int index = 0 , c = 0; 

while (index < NUM__ITEMS) { 
buffer [index] = Produce (); 
5 index = index + 1; 

C = C + 1 ; 

if (c == blockSize) { 

Increment (count, blockSize) ; 
c = 0; 

10 } 

} 

} 

ConsumerThread (int blockSize) 
15 { 

int index = 0, c = blockSize; 
while (index < NUM_ITEMS) { 
if (c == blockSize) { 

Check(count / index + blockSize); 
20 c = 0; 

n } 

,: ~ Consume (buffer [index] ) ; 

: f index = index + 1; 

i: ~ C = C + 1 ; 

m } 

M } 

i;n After writing a block of items to the buffer, the producer 

« thread increments the s-counter. Before reading a block of items 
;30 from the buffer, a consumer thread checks the counter. If the 
! ^ next block of items has not yet been written to the buffer, the 

consumer thread suspends until enough items have been written. 

The program does not require that the producer and consumer 

threads all use the same blockSize values. 
3 5 The monotonicity of counters helps guarantee deterministic 

synchronization and the equivalence of multithreaded and 

sequential execution . 

If shared variables are guarded against concurrent 

operations, a program that uses only counter synchronization can 



- 20 - 



Attorney 




;et No. 06618/389001/CIT- 



produce deterministic results on all executions. Moreover, if 
sequential execution of the program (i.e., execution ignoring 
the multithreaded keyword) does not deadlock, multithreaded 
execution is guaranteed not to deadlock and to produce the same 
5 results as sequential execution. These properties are extremely- 
useful in the testing and debugging of multithreaded programs. 

Even in the absence of concurrent operations on shared 
variables, traditional synchronization mechanisms can introduce 
nondeterminacy into a program through timing dependent race 

LP conditions between threads. For example, consider the following 

'S program that uses a lock: 

M multithreaded { 

/.£ { AcquireLock (&xLock) ; x = x+1; ReleaseLock ( fcxLock) ; } 

!:» { AcquireLock (fcxLock) ; x = x*2; ReleaseLock ( fcxLock) ; } 

W } 

; : = Even though there are no concurrent operations on x, the 

[2 resulting value of x is nondeterministic because of the race 

condition on the order in which the two threads acquire the 
20 lock. In contrast, because counters are monotonic, once a 

synchronization condition is enabled it remains enabled, and 
there is no possibility of a race condition to catch or miss a 
particular counter value. For example, consider the following 
program that uses a counter: 



25 



multithreaded { 

| CheckCounter ( &xCount , 0); x = x+1; IncrementCounter ( fcxCount , 1) 
{ CheckCounter (&xCount, 1); x = x*2; IncrementCounter ( &xCount , 1) 



: J 



- 21 - 



Attorney' ^^^Bket No. 06618/389001/CIT- 



The resulting value of x is deterministic, because the 
CheckCounter operations will succeed in the same order in all 
executions, therefore the operations on x will occur in the same 
order. Moreover, since sequential execution does not deadlock, 
5 multithreaded execution cannot deadlock and will always produce 
the same results as sequential execution. 

Programs that use only counter synchronization can still be 
erroneously nondeterministic if they do not guard against 
concurrent access to shared variables. For example, consider the 
XP following program using a counter: 

!: f multithreaded { 

in | CheckCounter (fcxCount, 0); x = x+l; IncrementCounter ( &xCount , 1); } 

;\ { CheckCounter (fcxCount, 0); x = x*2; IncrementCounter ( fcxCount , 1); } 

!;U The result of the program is nondeterministic because of 

^ the possibility of concurrent execution of the operations on x. 
\* The nondeterminacy is caused by concurrent access to a shared 
L 3 variable, not by a synchronization race condition. 
20 As a simple example, consider the following program to sum 

the elements of a two-dimensional array: 

void SumElements (float A [N] [N] , float *sum, int numThreads) 

{ 

int i ; 

25 SthreadCounter counter; 

SthreadCounterlnitialize (^counter) ; 

#pragma multithreadable mapping (blocked (numThreads) ) 
for (i = 0; i < N; i++) { 
3 0 int j; 

float rowSum; 

rowSum =0.0; 

for (j = 0; j < N; j++) 
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rowSum = rowSum + A[i] [j] ; 
SthreadCounterCheck (^counter, i) ; 
*sum = *sum + rowSum; 

SthreadCotmter Increment (^counter , 1) / 



5 



SthreadCounterFinalize (^counter) ; 



Without the counter operations, multithreaded execution of 



10 the for loop would not be equivalent to sequential execution, 
because the iterations all modify the same *sum variable. 
However, the counter operations ensure that multithreaded 
execution is equivalent to sequential execution. In sequential 
execution, the iterations are executed in increasing order and 

■i5 the SthreadCounterCheck operations succeed without suspending. In 

; := multithreaded execution, the counter operations ensure that the 
operations on *sum occur atomically and in the same order as in 

^ sequential execution. Iteration i suspends at the 

iu SthreadCounterCheck operation until iteration i - 1 has executed the 
!S 2 0 SthreadCounter Increment operation. 

U Conditions are carved out to prevent concurrent access to 

shared variables using counters. Essentially, each pair of 
operations on a shared variable must be separated by a 
transitive chain of counter operations. If these conditions can 
25 be shown to hold in any one execution of the program, they must 
hold in all executions of the program. Therefore, if sequential 
execution satisfies the conditions, multithreaded execution is 
also guaranteed to satisfy the conditions, hence produce the 
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same results as sequential execution. This result forms the 
basis of a powerful methodology for developing multithreaded 
programs using -sequential reasoning, testing, and debugging 
techniques . 

5 All the programs using counters so far discussed satisfy 

the conditions on shared variables, therefore are guaranteed to 
be deterministic. In addition, the program examples described 
herein have equivalent multithreaded and sequential execution. 
The cost of increased determinacy is decreased concurrency. 
XQ Synchronization using counters provides an effective means of 

'"'i controlling this tradeoff between determinacy and concurrency. 

in 

i± Counters can also be used as a stronger form of lock 

i;n synchronization, providing sequential ordering in addition to 
>■ mutual exclusion on a critical section. With the traditional 
IS implementation of mutual exclusion using a pair of lock 
^ operations, the order in which threads enter the critical 
^ section is nondeterministic . This is desirable in terms of 
maximizing concurrency, but is undesirable in terms of 
reasoning, testing, and debugging, and simply might not satisfy 
20 the desired program specification. Replacing the pair of lock 
operations with a pair of counter operations can guarantee 
deterministic results, at the cost of decreased opportunities 
for concurrency. 
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Consider the computation of a result object formed by- 
accumulating a series of independent subresults that are 
computed concurrently. For example, the result object could be a 
linked list and the accumulate operation could be an append, or 
5 the result object could be a summation and the accumulate 

operation could be an addition. Mutual exclusion is required to 
prevent interference between multiple concurrent accumulate 
operations on the result object. 

The following program implements the computation with one 
jbO thread computing each subresult, and a pair of lock operations 
:-.Z to provide mutual exclusion: 

; w Composite I tern result; 

i ! * Lock resultLock; 

;45 InitializeLock (&resultLock) ; 

h ~ multithreaded for (i = 0; i < N; i++) { 

; = Singleltem subresult = compute (i); 

M AcquireLock (fcresultLock) ; 

y}\ accumulate ( &result , subresult); 

:20 ReleaseLock (fcresultLock) ; 

M } 

;; - FinalizeLock ( &resultLock) ; 

Q Only one thread can hold resultLock at any given time, 

25 thereby ensuring mutual exclusion of the accumulate operations. 
However, if the accumulate operation is not associative and 
determinacy of results is desired, some other mechanism is 
required to ensure sequential (or at least deterministic) 
ordering, in addition to mutual exclusion. For example, neither 
30 appending an item to a linked list, nor floating point addition 
are associative operations. With both these examples, the 
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above program may produce different results on repeated 
executions . 

The following program implements the computation with the 
pair of lock operations replaced with a pair of counter 
operations, to provide both mutual exclusion and sequential 
ordering : 

Compositeltem result; 
Counter resultCount; 

InitializeCounter ( fcresultCount) ; 
multithreaded for (i = 0; i < N; i++) { 

Singleltem subresult = compute (i); 

CheckCounter (fcresultCount , i) ; 

accumulate (fcresult, subresult) ; 

IncrementCounter (&resultCount , 1) ; 

} 

FinalizeCounter (fcresult Count) ; 

As with the lock program, only one accumulate operation can 
execute at any given time. However, the accumulate operations 
are now additionally constrained to execute in sequential order. 
resultCount [i] = i indicates that thread i-l has completed its 
accumulate operation. The counter program has greater 
determinacy at the cost of less concurrency. With the lock 
program, an accumulate operation can execute concurrently with 
compute operations in all other threads. With the counter 
program, an accumulate operation can only execute concurrently with 
compute operations in higher numbered threads. 

The optimal tradeoff between determinacy and concurrency 
has to be made on a case by case basis. Counters are a powerful 
mechanism for providing sequential ordering on top of mutual 
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exclusion in the many cases where determinacy is important and 
the performance consequences of less concurrency are not great. 



The Sthreads Interface 
5 The code produced according to the present application can 

be expressed using the Multithreaded C pragma notation. As 
described previously, there is a direct correspondence between 
the pragma notation for thread creation and the Sthreads library 
functions that support thread creation. As a simple example, the 
j!gO following is a program implemented using Sthreads: 



i : H typedef struct { 
i'V float (*A) [N] ; 

I . float *sum; 

!.~ SthreadCounter * counter; 

15 } LoopArg s ; 

^ void LoopBody{int i, int notusedl, int notused2., LoopArgs *args) 

M { 

ry int j ; 

1 20 float rowSum; 

rowSum = 0.0; 
"Z t for (j = 0; j < N; j++) 

rowSum = rowSum + (args->A) [i] [j] ; 
SthreadCounterCheck (args- >counter, i) ; 
25 *(args->sum) = *(args->sum) + rowSum; 

SthreadCounter Increment (args- >counter, 1) ; 



void SumElements (float A [N] [N] , float *sum, int numThreads) 
30 { 

int i ; 

SthreadCounter counter; 
LoopArgs args; 

3 5 SthreadCounterlnitialize (^counter) ; 

args. A = A; 

args . sum = sum ; 

args. counter = &counter; 

SthreadRegularForLoop ( 

4 0 (void (*) (int, int, int, void *)) LoopBody, 

(void *) &LoopArgs, 

0, STHREAD_CONDITION_LT, N, 1, 
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1, STHREAD_MAPP I NG_B LOCKED , numThreads, 

STHREAD_PRIORITY_PARENT, STHREAD_STACK_S I ZE_DE FAULT ) ; 
SthreadCounterFinalize (fccounter) ; 

} 

Although this program is syntactically more complicated 
than the Multithreaded C version, it is considerably less 
complicated than the same program expressed using Windows NT 
threads. The mechanics of creating threads, assigning iterations 
to threads, and waiting for thread termination is handled within 
the Sthreads library call. 

The Sthreads multithreaded programming system is 
implemented as a transparent add-on to an existing program 
development system, e.g., a compiler or interpreter, or other 
program development environment. The notation and implementation 
allows multithreaded and multithreadable code constructs to be 
directly translated into a high-level structured thread library. 
This translation is implemented as a preprocessor that can be 
transparently called prior to the standard compilation phase in 
an existing program development system. 

For example, when integrated with the Microsoft Visual C++ 
programming system, the standard CL (Compiler-Linker) is 
replaced by a special Sthreads tool that calls the Sthreads 
preprocessor on program files, then calls the standard (renamed) 
Visual C++ CL. 
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Integration of Sthreads with existing programming systems 
allows programmers new flexibility without adopting new 
programming systems to use the power of multithreading. They can 
use their standard editor, debugger, compiler, etc., and simply 
5 add Sthreads to the system. It also means that Sthreads 

piggybacks on the quality of code generation and error analysis 
of the underlying development system. 

Preprocessing had previously been used for many kinds of 
program "source-to-source" transformations. Sthreads in 
jifeO contrast, implements a full-fledged, sophisticated multithreaded 
s: f programming system by using a preprocessing integrated with a 
M standard program development environment. 

^ One implementation has been created in the ANSI C language, 

f thereby defining a "Multithreaded C" language. A structured 
j 3-5 thread library (Sthreads) is called by the languages. In both 
Multithreaded C and Sthreads, thread creation constructs are 
multithreaded variants of sequential "block" (i.e., sections of 
program code distinctly defined by conventional program 
constructs) and "for loop" constructs. In the Multithreaded C 
20 implementation, these constructs are supported as pragma 

annotations to a sequential program. With Sthreads, exactly the 
same constructs are supported as library calls. At both levels, 
synchronization objects and operations are supported as Sthreads 
library calls. 
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In this embodiment, the Sthreads library for Windows NT can be 
implemented as a very thin layer on top of the Win3 2 thread API . 
As a consequence, there is essentially no performance overhead 
associated with using Sthreads or Multithreadable C, as compared 
5 to using Win32 threads directly. 

Multithreaded C is implemented as a portable source-to- 
source preprocessor that directly transforms annotated blocks 
and for loops into equivalent calls to the Sthreads library. The 
programmer has the option of either using the pragma annotations 
(H) and preprocessor or making Sthreads calls directly. The Sthreads 
s: * library and Multithreaded C preprocessor are integrated with 
M Microsoft Developer Studio Visual C++. Building a project 
^ preferably automatically invokes the preprocessor where 
f necessary and links with the Sthreads library. 

;15 Multithreadable blocks and for loops are implemented as a 

sequence of CreateThread calls followed by a WaitForSingleObject call 
on an event . Terminating threads perform an InterlockedDecrement 
call on a counter, and the last thread to terminate performs a 
SetEvent call on the event. Flags are implemented directly as 
20 Win32 events. Counters are implemented as linked lists of Win32 
events, with one event for every value on which some thread is 
waiting. Locks are implemented directly as Win32 critical 
sections. Barriers are implemented as a pair of Win32 events and 
a Win32 critical section. 
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An important issue of the multithreading operation comes 
about when considering multiple processors. The hardware and 
operating systems of modern technology allow for multiprocessor 
systems. Current operation in multiprocessor systems, however, 
5 have often simply operated on one but not the other processor. 
By multithreading in this way, the different threads can 
actually be executed on the different processors. 

In operation, when a multithreading indicator (such as a 
"compile as multithreaded" flag/button/environment -variable) is 
^0 set, both multithreadable and multithreaded blocks/loops are 

== compiled to multithreaded code. When the multithreading 

rn 

indicator is not set, the multithreaded blocks/loops are 

M 

!:n compiled to multithreaded code and the multithreadable 
« blocks/loops are compiled into ordinary sequential code. This 
jl.5 allows a programmer to mix constructs that only have a 
"2 multithreaded meaning (e.g., real-time control and systems 
programming uses of threads) with constructs that can be 
compiled into threads for multiprocessor performance or compiled 
into equivalent sequential code when developing and debugging. 
20 The invention allows a program to run as fast as a 

sequential program on one processor, but significantly faster on 
multiprocessors, without recompilation, relinking, or 
reconfiguration. The invention thus allows a program to adapt 
dynamically to changing resources. Use of monotonic flags and 
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monotonic counters makes embodiments of the invention reliable 
and timely. 

The mapping of Statements/Iterations onto threads is relatively 
simple. One thread is used for each statement /chunk, or for a 
5 small number of statements/chunks. 

A Typical for loop may have thousands or millions of 
iterations. The Overhead associated with assigning units of work 
to threads is significant. The present application defines 
assigning the iterations in contiguous "chunks" . Significant 
ftO unit of work should be performed by each chunk. 

i: z For example : 

j.n 

M ft PRAGMA MULTITHREADABLE CHUNKSIZE ( 1 0 0 0 ) , MAPPING (BLOCKED ( T) ) 

lip FOR (i = 0; I < N; I + +) 

A [I] = B[l] ; 

l.S. 

"Z, Interaction of Chunksize and Mapping is described in the 

following example: 

ft PRAGMA MULTITHREADABLE CHUNKSIZE (2) , MAPPING (BLOCKED (4 ) ) 
20 FOR (I = 50; I >= 10; I = I - 2) 

DOSOMETHING ( I ) ; 

The Complete Sthreads Library includes a number of statements: 
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Processor Management : SthreadsGetNumProcessorsPresent, 



SthreadsSetNumProcessorsUsed , SthreadsGetProcessors Present , 



SthreadsSetProcessorsUsed . 



Thread Creation: SthreadsBlock , SthreadsRegularForLoop . 



5 



Thread Scheduling: SthreadsGetCurrentPriority, 



SthreadsSetCurrentPriority . 

Flags : SthreadsFlagInitialize, SthreadsFlagFinalize, SthreadsFlagSet, 
SthreadsFlagCheck , SthreadsFlagReset . 

Counters : SthreadsCounterInitialize, SthreadsCounterFinalize, 
iO Sthreads Counter Increment, SthreadsCounterCheck, SthreadsCounterReset . 
„= Locks : SthreadsLockInitialize, SthreadsLockFinalize, 

M SthreadsLockAcquire , SthreadsLockRelease . 



: * these constructs are set forth in the appendix. 

FIG. 4 is a process flowchart showing a method for 
compiling multithreadable code in accordance with one embodiment 
of the invention. The computer program source code text 400 

20 includes annotations defining multithreadable code constructs 
(and, optionally, multithreaded code constructs) and any 
necessary processor management, thread creation, and 
synchronization constructs (such as monotonic flags and 
counters) . If a multithreading indicator is set 401, the source 



Barriers : SthreadsBarrierInitialize, SthreadsBarrierFinalize, 



SthreadsBarrierPass , SthreadsBarrierReset . 



f £5 



Examples of computer program code implementing each of 
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code text 400 is processed by a pre-processor 402 that parses 
the source into an expanded computer program text . The expanded 
computer program text includes inserted calls to an Sthreads 
library 406 to invoke multithreaded program operations wherever 
5 a source code annotation called for multithreadable 

functionality. A conventional compiler 406 then communicates 
with a linker 410 which links pre-existing routines from the 
Sthreads library 4 06 with the output of the compiler to create 
an executable module 412. 
X0 If the multithreading indicator is not set 401, the 

= original computer program source code text 400 is compiled and 
L linked in conventional fashion, with each section of 
j;n multithreadable code constructs compiled as sequentially 

; : ^ 
si 

executing code. Annotations not recognized by the compiler 408 

f'&5 . are ignored. 

A convenient implementation shortcut that permits ready use 
of conventional compilers and linkers is to rename a pre- 
existing compiler- linker executable file to a new name, and 
assign the old name of the compiler-linker executable file to 

20 the pre-processor. The pre-processor then can call the compiler- 
linker executable file when needed by invoking the new name. 
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Synchronization Using Locks 

Locks are provided to express nondeterministic 
synchronization, usually mutual exclusion, within multithreaded 
blocks and for loops . Sthread locks support the usual Acquire and 
5 Release operations. The order in which concurrent Acquire 

operations succeed is nondeterministic. Therefore, there is very 
little use for locks within multithreadable blocks and for loops. 
As a simple example, consider the following program to sum the 
elements of a two-dimensional array: 



li> void SumElements (float A [N] [N] , float *sum, int numThreads ) 

*- { 

!.I i n t i ; 

i- M SthreadLock lock; 

13*5 SthreadLocklnitialize (&lock) ; 

|;n #pragma multithreaded mapping (blocked (numThreads) ) 

H f 03T (i = 0; i < N; i+ + ) { 

, int 3 ' 

float rowSum; 

:2 0 rowSum = 0.0; 

J y for (j = 0; j < N; j++) 

V a rowSum = rowSum + A[i] [j] ; 

SthreadLockAcquire ( &lock) ; 

l : 3 *sum = *sum + rowSum; 

:25 SthreadLockRelease (&lock) ; 

} 

SthreadLockFinalize (&lock) ; 



30 Like the flag operations in the program, the lock 

operations in this program ensure that the operations on *sum 
occur atomically. However, unlike the flag operations, the lock 
operations do not ensure that the operations on *sum occur in the 
same order as in sequential execution, or even in the same order 
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each time the program is executed. Therefore, since floating- 
point addition is not associative, the program may produce 
different results each time it is executed. However, because 
execution order is less restricted, this program allows more 
5 concurrency than the program described above. This is an 

example of the commonly-occurring tradeoff between determinacy 
and efficiency. 

Synchronization Using Barriers 
rJO S-thread barriers are provided to express collective 

S! ~ synchronization of a group of threads in cases when thread 
u termination and recreation is too expensive. The barriers 
i;n described herein support the usual Pass operation. All the 

s" ~ 

threads in a group must enter the Pass operation before all the 
JKL5 threads in the group are allowed to leave the Pass operation. In 
current systems, the cost of N threads executing a Pass operation 
is less than the cost of creating and terminating N threads. 
Therefore, a typical use of barriers is to replace a sequence of 
multithreadable loops with a single multithreaded loop 
20 containing a sequence of barrier Pass operations. However, with 
modern lightweight thread systems such as Windows NT, we are 
discovering that barriers are required for efficiency in very 
few circumstances. 

A number of examples are described herein. 
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Trivial Example: Independent Iterations 

INT ARRAYSUM ( FLOAT A [N] [N] ) 

/* Sums the elements of a 2 -dimensional array. */ 

{ 

5 FLOAT SUM, ROWSUM[N] ; 

INT I; 

SUM = 0.0; 

# PRAGMA MULT I THREADABLE 
10 FOR (1=0; I < N; i + +) { 

INT J; 

ROWSUM[l] = 0.0; 

FOR ( J = 0 ; J < N; J++) 

rowSum[i] = rowSum[i] + A[i] [j] ; 

15 } 

FOR (I = 0; I < N; I + +) 

SUM = SUM + ROWSUM[l] ; 
*.« RETURN SUM; 

" } 
Jo 

s n 

U A more difficult example is shown in the following. 



; - : Incorrect Example : Nondeterminacy 

int ArraySum ( float A [N] [N] ) 
* ; 45 /* Sums the elements of a 2 -dimensional array. */ 

K { 

"Z FLOAT SUM; 

INT I; 

SthreadsLock SUMLOCK; 

30 

SthreadsLockInitialize (&SUMLOCK) ; 
SUM = 0.0; 

# pragma multithreadable 

FOR (I = 0; I < N; I++) { 
35 INT J; 

FLOAT ROWSUM = 0.0; 

FOR (J = 0 ; J < N; J++) 

ROWSUM = ROWSUM + A[l] [j] ; 

SthreadsLockAcquire (&SUMLOCK) ; 
4 0 SUM = SUM + ROWSUM; 

SthreadsLockRelease (&SUMLOCK) ; 

} 

SthreadsLockFinalize (&SUMLOCK) ; 
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RETURN SUM; 

} 

int Array Sum ( float A[N] [N] ) 

/* Sums the elements of a 2 -dimensional array. */ 
5 { 

float sum; 

INT I; 

S threads Counter sumCount; 

10 SthreadsCounterInitialize (&SUMCOUNT) ; 

SUM = 0.0; 

# PRAGMA MULTITHREADABLE 
FOR (I = 0; I < N; I + + ) { 
INT J; 

1 5 FLOAT ROWSUM = 0.0; 

FOR (j = 0; J < N; J++) 

ROWSUM = ROWSUM + A[l] [j] ; 

SthreadsCounterCheck(&sumCount, i) ; 
sum = sum + rowsum; 
!20 SthreadsCounter Increment ( &sumCount , 1 ) ; 

:= } 

j;n SthreadsCounterFinalize (&sumCount) ; 

M RETURN SUM; 

M 

l;35 As can be seen, iterations cannot be executed as separate 

== threads because of nondeterminacy in the top. However, the 

counters allow determinacy between the system therefore enabling 
""Z the system to be multithreaded. 



30 Single-Writer Multiple-Reader Broadcast 

Counters can be used to provide elegant, flexible, and 
efficient dataflow synchronization between a single writer and 
multiple readers of a sequence of items written to an array. In 
this synchronization pattern, reading an item does not remove it 

35 from the sequence— each reader independently reads the entire 

shared array. Because a counter has multiple thread suspension 
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queues, a single counter object can be used to synchronize the 



writer thread and any number of completely independent reader 



threads, with each thread potentially having a different 



granularity of synchronization. The writer thread incrementing 



5 the counter broadcasts the availability of data to the entire 



set of reader threads. 



The following program demonstrates the single-write 



multiple-reader broadcast pattern with synchronization on every 



item: 



ffijO void Writer (Item *data, int n, Counter * dataCount) 

3 { 

~ int i; 

;;~ for (i = 0; i < n; i++) { 

ijl data[i] = Generate I tern (i) ; 

ik5 IncrementCounter (dataCount , 1) ; 



iin 

void Reader (Item *data, int n, Counter *dataCount) 

!m { 

|2 0 int i; 

M for (i = 0; i < n; i++) { 

CheckCounter (dataCount , i + 1) ; 
; " Useltem (data [i] ) ; 

Ss } ! 

w Item data [N] ; 

Counter dataCount; 
int r ; 

30 

InitializeCounter (&dataCount) ; 
multithreaded { 

Writer (data, N, dataCount); 

multithreaded for (r = 0; r < numReaders; r++) 
3 5 Reader (data, N, dataCount); 

FinalizeCounter (&dataCount) ; 



4 0 One Writer thread and an arbitrary number of Reader threads 



are executed concurrently, with communication through the shared 



data array, and synchronization through the dataCount counter. At 
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any point, some Reader threads may be suspended in their 
CheckCounter operation, waiting for the Writer thread to increment 
dataCount, while other Reader threads may be reading data items 
that have previously been written. The Reader threads execute 
5 independently of each other and do not synchronize their actions 
in any manner. The synchronization pattern is strictly a one-to- 
many broadcast from the Writer thread to the Reader threads . 

Synchronization on every item that is written and read may 
be too expensive if the time taken to generate and use an item 

10 is too small. The single-reader multiple-writer broadcast 
2 pattern can be generalized to allow the writer and each reader 
thread to synchronize on a block of items instead of on 

rn individual items. The following program adds an individual 

» granularity of blocked synchronization to the writer and each 

ilj5 reader thread: 

['2 void Writer (Item *data, int n, Counter dataCount, int blockSize) 

{ int i; 
\,2 for (i = 0; i < n; i++) { 

data[i] = Generateltem { i) ; 
20 if ( (i+i) %blockSize == 0) 

IncrementCounter (dataCount , blockSize) ; 

IncrementCounter (dataCount , n- (n/blockSize) blockSize) ; 

25 void Reader (Item *data, int n, Counter *dataCount, int blockSize) 

int i ; 

for (i = 0; i < n; i++) { 
if (i%blockSize == 0) 
30 CheckCounter (dataCount , min (i+blockSize, n) ) ; 

Useltem (data [i] ) ; 
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The Writer and Reader threads now increment and check the 
dataCount counter in multiples of blockSize and write and read the 
data array in blocks of items. There is no requirement that 
blockSize be the same in all threads. Different threads can be 

5 passed different blockSize based on their individual performance 
characteristics and requirements. This pattern is now extremely 
flexible and easily adaptable with regard to practical 
performance tuning . 

The single-writer multiple-reader broadcast pattern is a 

0 dataflow synchronization pattern that occurs in many diverse 

applications of threads to multiprocessing. For example, in the 
Paraffins Problem, an array of molecules of a certain size can 
be generated by one thread and concurrently read by other 
threads that in turn generate arrays of larger molecules. The 

5 pattern is very different from, for instance, the multiple- 
writers multiple-readers bounded-buffer problem, which is 
elegantly solved using semaphores. Just as counters are not well 
suited to implementing bounded buffers, semaphores and other 
traditional synchronization mechanisms are not well suited to 

0 implementing the single-writer multiple-reader broadcast 
pattern . 
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Another Example Application: Aircraft Route Optimization 

The Aircraft Route Optimization Problem is part of the U.S. 
Air Force Rome Laboratory C3I Parallel Benchmark Suite. For this 
application, we achieved better performance using Sthreads on a 
5 quad-processor Pentium Pro system running Windows NT than the 
best reported results for message-passing programs running on 
expensive Cray and SGI supercomputers with up to 64 processors. 
The flexibility of shared-memory, lightweight multithreading, 
and sequential development methods allowed us to develop a much 

AO more sophisticated and efficient algorithm than would be 

:: r possible on a message-passing supercomputer. 

i'n 

U The C3I Parallel Benchmark Suite 

M 

!:fi The U.S. Air Force Rome Laboratory C3I Parallel Benchmark 

i! Suite consists of eight problems chosen to represent the 

!l5 essential elements of real C3I (Command, Control, Communication, 

and Intelligence) applications. Each problem consists of the 

following: 

A problem description giving the inputs and required 
outputs . 

20 An efficient sequential program (written in C) to solve the 

problem. 

The benchmark input data. 

A correctness test for the benchmark output data. 
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For some of the problems, a parallel message-passing 
program is also provided. Rome Laboratory maintains a publicly 
accessible database of reported performance results. 

The C3I Parallel Benchmark Suite provides a good framework 
5 for evaluating our structured multithreaded programming system. 
The problems are computationally intensive and involve a variety 
of complex algorithms and data structures. The sequential 
program provides us with a good starting point and a fair basis 
for performance comparison. The performance database allows us 
r i0 to compare our results with those of other researchers. For 
^ these reasons, we are developing multithreaded solutions to 
.U several of the C3I Parallel Benchmark Suite problems. 
|;n The task in the Aircraft Route Optimization Problem is to 

* find the lowest-risk path for an aircraft from an origin point 

| M15 to a set of destination points in the airspace over an uneven 
s: i; terrain. The risk associated with each transition in the 

airspace is determined by its proximity to a set of threats. The 
problem involves realistic constraints on aircraft speed and 
maneuverability. The aircraft is also constrained to fly above 
20 the underlying terrain and beneath a given ceiling altitude. 

The problem is essentially the single-source, multiple- 
destination shortest path problem with a large, sparsely 
connected graph. The airspace for the benchmark is 100 km by 100 
km in area and 10 km in altitude, discretized at 1 km intervals. 
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The 100,000 positions in space correspond to 2,600,000 nodes in 
the graph, since each position can be reached from 26 different 
directions. Because of aircraft speed and maneuverability 
constraints, each node is connected to only nine or ten 
5 geographically adjacent nodes. Therefore, the graph consists of 
approximately 2.6 million nodes and 26 million edges. 

The sequential algorithm to solve the Aircraft Route 
Optimization Problem is based on a queue of nodes. Initially the 
queue is empty except for the origin node. At each step, one 

X0 node is removed from the queue. Valid transitions from this 

i.3 

source node to all adjacent destination nodes are considered. 
U For each destination node, if the path to the node via the 

|;fi source node is shorter than the current shortest path to the 

U 

« node, the path to the node is updated and the node added to the 

I'D 5 queue. The algorithm continues until the queue is empty, at 

s: = which stage the shortest paths to all reachable nodes have been 

T. -J 

computed. 

The queue is ordered on path length so that shorter paths 
are expanded before longer paths. This has a significant effect 
2 0 on performance. Without ordering, longer paths are expanded, 
then discarded when shorter paths to the same points are 
expanded later in the computation. However, whether the queue is 
ordered, partially ordered, or unordered does not affect the 
results of the algorithm. 
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The most straightforward approach to obtaining parallelism 
in the Aircraft Route Optimization Problem is to geographically 
partition the airspace into blocks, with one thread (or process) 
responsible for each block. Each thread runs the sequential 
5 algorithm on its own block using its own local queue and 

periodically exchanges boundary values with neighboring blocks. 
This approach is particularly appealing on distributed-memory , 
message-passing platforms, because memory can be permanently 
distributed according to the blocking pattern. If the threads 

,i0 execute a reasonably large number of iterations between boundary 
exchanges, good load balance can be achieved. 

M The problem with this algorithm is that, as the number of 

i : 

|:n blocks/threads is increased the total amount of computation also 
« increases. Therefore, any speedup is based on an increasingly 
|il5 inefficient underlying algorithm. At any time, the local queues 
:: : in most blocks contain paths that are too long and are 

irrelevant to the actual shortest paths. The processors are kept 
busy performing computation that is later discarded. At any 
given time, it is only productive to work on an irregular and 
20 unpredictable subset of the graph. However, irregular and 

adaptive blocking schemes do not solve the problem, since there 
is usually equal work available in all blocks. The issue is the 
distinction between productive and unproductive work. 
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Our solution is to statically partition the airspace into a 
large number of blocks and to use a much smaller number of 
threads. A measure of the average path length is maintained with 
each local queue. At each step, the blocks with local queues 
5 containing the shortest paths are assigned to the threads. 
Therefore, the subset of blocks that are active and the 
assignment of blocks to threads change dynamically throughout 
program execution. This algorithm takes advantage of the 
symmetric multiprocessing model, in which all threads can access 
,1-0 the entire memory space with uniform cost. It also takes 
S advantage of the lightweight multithreading model to achieve 
U good load balance, since the workload within each thread at each 

j.JL 

|;n step is highly variable. 

« The ability to develop, test, and debug using sequential 

I M 5 methods was crucial in the development of this sophisticated 
multithreaded algorithm. The entire program was tested, and 

' :=i debugged in sequential mode before multithreaded execution was 
attempted. In particular, development of the complex boundary 
exchange and queue update algorithms would have been 

20 considerably more difficult in multithreaded mode. 

The ability to analyze and tune performance using 
sequential methods was also very important. Good performance 
depended on exposing enough parallelism without significantly 
increasing the total amount of computation. We determined 
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efficient values for the number of blocks, the number of 
threads, and the number of iterations between boundary exchanges 
by measuring computation times and operation counts of the 
multithreaded program in running in sequential mode. This 
detailed analysis would have been very difficult to perform in 
multithreaded mode. We avoided memory contention in 
multithreaded mode by avoiding cache misses in sequential mode. 
The analysis of memory access patterns in sequential mode is 
much simpler than in multithreaded mode. 




All Pairs Shortest Paths Example 

This example describes the algorithmic and performance 
advantages of counter synchronization. In the example, a counter 
is used as a less restrictive, and consequently more efficient, 
replacement for a barrier. The example program is a 
multithreaded solution to the all-pairs shortest-path problem 
using the Floyd-Warshall algorithm. Using traditional 
synchronization mechanisms, this problem can be solved using one 
barrier or, more efficiently, an array of condition variables. 
We show how the efficient solution can be implemented using a 
single counter instead of an array of condition variables. We 
give timing measurements comparing the performance of the 
barrier, condition variable, and counter algorithms. 
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The all-pairs shortest-path problem takes as input the 
edge-weight matrix of a weighted directed graph, and returns the 
matrix of shortest -length paths between all pairs of vertices in 
the graph. The graph is required to have no cycles of negative 
length, and the weight of the edge from a vertex to itself is 
required to be zero. 

The following program solves the all-pairs shortest-path 
problem using the sequential Floyd-Warshall algorithm: 

VOID SHORTESTPATHSl ( INT EDGE [N] [N] , INT PATH [N] [N] ) 

INT K, I, J; . 

PATH [0 . .N-l] [0 . .N-l] = EDGE [ 0 . . N- 1 ] [ 0 . . N- 1 ] ; 
FOR (K = 0; K < N; K++) 

FOR (i = 0; I < N; I + + ) 
FOR (j = 0; J < N; J++) { 

INT NEWPATH = PATH[l] [k] + PATH [k] [j] ; 
IF (NEWPATH < PATH[l] [j] ) PATH[l] [j] = NEWPATH; 

} 1 

Initially, path[i] [j] is assigned edge [i] [j] , for all i and j. 
(For brevity, we use a notational shorthand for array 
assignment.) After the kth iteration, path[i] [j] is the shortest 
path from vertex i to vertex j with intermediate vertices only in 
vertices 0 to k. Therefore, after N iterations, path[i] [j] is the 
shortest path from vertex i to vertex j with no restrictions on 
the intermediate vertices. 

The following program solves the all-pairs shortest path 
problem using a multithreaded version of the Floyd-Warshall 
algorithm, with a barrier for thread synchronization: 



- 48 - 



Attorney's t No. 06618/389001/CIT- 

void Shortest Paths2 (int edge [N] [N] , int path [N] [N] , int numThreads ) 

int t; 
Barrier b; 

5 

path[0. .N-l] [0. .N-l] = edge [0. .N-l] [0. .N-l] ; 
InitializeBarrier (&b, numThreads) ; 
multithreaded for (t = 0; t < numThreads; t++) { 
int k, i, j ; 
10 for (k = 0; k < N; k++) { 

for (i = t*N/numThreads; i < ( t+1) *N/numThreads ; i++) 
for (j = 0; j < N; j+ + ) { 

int newPath = path[i] [k] + path [k] [j] ; 

if (newPath < path[i] [j]) path[i] [j] = newPath; 

PassBarrier (&b) ; 

) > 

FinalizeBarrier ( &b) ; 

20 } 

The multithreaded outer loop creates numThreads threads. Each 
;:== thread executes the N iterations of the Floyd-Warshall algorithm 

on a subset of the rows of the path matrix. To keep the 
1=25 iterations synchronized, the threads pass through an N-way 
|;n barrier at the end of each iteration. There are no sharing 
« violations on the concurrent accesses to path across the threads, 
because the algorithm will never assign to path[i] [k] or path [k] [j] 
during iteration k. 
: *$0 The barrier algorithm successfully divides the work among 

an arbitrary number of threads. However, in requiring that all 
threads complete each iteration before any thread begins the 
next iteration, the algorithm does not express the full 
opportunities for concurrency inherent in the data dependencies. 
35 As a consequence, the program is less than optimally efficient. 
N-way synchronization at the barrier is a bottleneck that 
creates delays on entry and exit, and processor load imbalance 
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can occur if all threads do not reach the barrier 
simultaneously . 

A More Efficient Multithreaded Solution Using Condition Variable 
Synchroni zat ion 

5 The following program solves the all -pairs shortest path 

problem using a more, efficient multithreaded version of the 
Floyd-Warshall algorithm, with an array of N condition variables 
for thread synchronization: 

void ShortestPaths3 (int edge [N] [N] , int path [N] [N] , int numThreads) 
10 { 

int k, t; 
□ Condition kDone [N] ; 

i 2 int kRow[N] [N] ; 

;;|5 path[0. .N-l] [0. .N-l] = edge [0 . .N-l] [0 . .N-l] ; 

lii for (k = 0; k < N; k++) InitializeCondition ( fckDone [k] ) ; 

U kRow[0] = path[0] [0. .N-l] ; 

I , SetCondition (&kDone [0] ) ; 

* : ~ multithreaded for (t = 0; t < numThreads; t++) { 

i;20 int k, i ( j; 

j;3 for (k = 0; k < N; k+ + ), { 

CheckCondition (&kDone [k] ) ; 
i: for (i = t*N/numThreads ; i < ( t+1) *N/numThreads ; i++) { 

M for (j = 0; j < N; j++) { 

|2 5 int newPath = path[i] [k] + kRow[k] [j] ; 

■ ^ if (newPath < path[i] [j]) path[i] [j] = newPath; 

i: ; if (i == k+1) { 

kRow[k+l] [0. .N-l] = path [k+1] [0. .N-l] ; 
: 30 SetCondition(&kDone [k+1] ) ; 

- , l::.L~, 

As with the barrier algorithm, each thread executes the N 
iterations of the Floyd-Warshall algorithm on a subset of the 
4 0 rows of the path matrix. However, each thread can individually 
continue with its next iteration as soon as the necessary data 
is available, instead of waiting for the previous iteration to 
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complete in all the other threads. Condition variable kDone [k] is 
set when row k of the path matrix has been computed in iteration 
k-1. Each thread waits on kDone [k] before executing iteration k. 
To avoid sharing violations, row k of the path matrix computed in 
5 iteration k-1 is stored in kRow[k] . 

The condition variable algorithm avoids the inefficiencies 
associated with barrier synchronization. Threads synchronize 
individually, rather than in an N-way bottleneck, and faster 
threads can execute many iterations ahead of slower threads. 
±10 Potentially, the N threads can be executing in up to N different 
~ iterations. One extra cost of this algorithm is the storage for 
U the kRow matrix. However, the most significant extra cost is 
i;n allocation of N condition variables. 

The following program solves the all -pairs shortest path 

: s 

jl.5 problem using the efficient multithreaded version of the Floyd - 
i'• ::::: 

% Z Warshall algorithm, with a single counter for thread 
,! " synchronization in place of N condition variables: 
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void ShortestPaths3 (int edge [N] [N] , int path [N] [N] , int numThreads) 

{ 

int k , t ; 
Counter kCount; 
5 int kRow[N] [N] ; 

path[0. .N-l] [0. .N-l] = edge[0. .N-l] [0. .N-l] ; 
InitializeCounter ( &kCount) ; 
kRow[0] = path[0] [0 . .N-l] ; 
10 multithreaded for (t = 0; t < numThreads; t++) { 

int k, i, j ; 

for (k = 0; k < N; k++) { 

CheckCounter ( &kCount , k) ; 

for (i = t*N/numThreads; i < (t+1) *N/numThreads ; i++) { 
15 for (j = 0; j < N; j++) { 

int newPath = path[i] [k] + kRow[k] [j] ; 

if (newPath < path[i] [j]) path[i] [j] = newPath; 

if (i == k+1) { 

20 kRow[k+l] [0. .N-l] = path [k+1] [0. .N-l] ; 

IncrementCounter (&kCount , 1); 

. , ■ ■ 1 

Final izeCounter (SckCount) ; 

Q } 

i'n Operations on N different values of the single counter 

Q0 replace operations on N different elements of the array of 
O condition variables. The algorithm has the same performance 
M advantages over the barrier algorithm, without the cost of 

statically allocating and maintaining N synchronization objects. 
^ Internally, the counter may create synchronization objects for 
3 5 the distinct counter values on which threads are suspended. 

However, in practice, the number of these objects in existence 
at any given time is likely to be a small fraction of N. 



Three Synchronization Patterns Example 
4 0 Three examples of practical synchronization patterns are 

described that can be expressed more elegantly (and often more 



- 52 - 



Attorney's t No. 06618/389001/CIT- 

efficiently) using counters than with traditional 
synchronization mechanisms. For each of these synchronization 
patterns, a small example program is provided to demonstrate the 
pattern and a description of the importance of the pattern to 
5 real problems. This is far from an exhaustive list of patterns 
to which counters can usefully be applied. Counters are equally 
applicable to many other situations, particularly dataflow style 
synchronization patterns arising in the 'application of threads 
to multiprocessing. 
iiO Counters can often be used to replace traditional barrier 

synchronization with a less restrictive form of "ragged" 
M barrier. With a ragged barrier, each thread waits at the barrier 
j;-n point only until its own individual data dependencies have been 

satisfied, instead of until the data dependencies of all threads 
ji.5 have been satisfied, as with a traditional barrier. We have 
l: Z already given one example of this pattern in Section 0, with the 
multithreaded Floyd-Warshall algorithm to solve the all-pairs 
shortest-path problem. In this section, we give another more 
straightforward example, based on boundary exchange in a time- 
2 0 stepped simulation. 

Consider a time-stepped simulation of a one-dimensional 
object subdivided into N cells. The state of internal cell i at 
time t is a function of the states of cells i-1, i, and i+1 at 
time t-1. The states of the leftmost and rightmost cells remain 
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constant over time. An example is simulation of heat transfer 
along a metal rod. Similar boundary exchange requirements occur 
in most multithreaded simulations of physical systems in one or 
more dimensions . These requirements are traditionally satisfied 
5 using barrier synchronization. 

The following program implements the simulation using one 
thread for each cell, with traditional barrier synchronization 
between threads before cell state exchanges and updates at each 
time step: 

i;|0 float state [N] ; 

,% Barrier b; 

state [o . . N- 1] = initial cell states 
|;n InitializeBarrier { &b, N-2) : 

\%5 multithreaded for(i = 1; i < N-l; i++) { 

float leftState, 1 rightState; 
M for (t = 1; t <= numSteps; t++) { 

j;fj PassBarrier ( &b) ; 

leftState = state [i-1] ; 
h &Q rightState = state [i + 1] ; 

n PassBarrier (&b) ; 

• ! S5 FinalizeBarrier ( &b) : 

--3 All threads synchronize at the barrier twice every time 

step: once before exchanging cell states, and again before 
updating cell states. However, complete barrier synchronization 
30 between all threads is unnecessarily restrictive. The conditions 
for safely exchanging and updating the cell states involve 
dependencies between pairs of neighboring cells, not across all 
cells. As a consequence of using barriers, the performance of 
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the program is potentially subject to synchronization bottleneck 
and load imbalance problems. 

The following program implements the same simulation using 
an array of counters to provide ragged barrier synchronization 
5 between threads : 

float state [N] ; 
Counter c [N] ; 

state [o . . n- l] = initial cell states; 
10 for (i = 0; i < N; i++) Counterlnitialise ( &c [i] ) ; 

Increment Counter (&c [0] , 2*numSteps) ; 
Increment Counter (&c [N-l] ) , 2*numSteps) ; 
multithreaded for(i =1; i < N-l; i++) { 

float leftState, rightState, myState = state [i] ; 
15 for (t = 1; t <= numSteps; t++) { 

CheckCounter (&c [i-1] , 2*t-2)); leftState = state [i-1]; 
CheckCounter (&c [i+1] , 2*t-2)); rightState = state [i+1]; 
f IncrementCounter (&c [i] , 1) ; 

k.2 myState = f (leftState, myState, rightState); 

.20 CheckCounter (&c [i-1] , 2*t-l) ; 

CheckCounter (&c [i+1] , 2*t-l) ; 
) state [i] = myState; 

I* IncrementCounter (&c [i] , 1); 

!25 ) 1 

^ for (i = 0; i < N; i++) FinalizeCounter ( &c [i] ) ; 

j is a As with the traditional barrier algorithm, the threads 

M synchronize every time step before exchanging cell states, and 
i30 again before updating cell states. However, the synchronization 
is between pairs of neighboring threads via an array of 
counters. c[i] = 2*t-1 indicates that thread i has finished 



reading both neighboring cell states in time step t, and c[i] = 
2*t indicates that thread i has completed time step t. Pairwise 
35 synchronization removes the synchronization bottleneck of a 

traditional barrier and reduces load imbalance by allowing some 
threads to execute ahead of other threads . The barrier could be 
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made even more ragged using separate counters to synchronize 
with left and right neighbors. 

The major cost in the implementation of ragged barriers 
using counters is the need for N counter objects instead of one 
5 barrier object. However, the number of counters needed is 

proportional to the number of threads, not to the problem size. 
This cost is unlikely to be a practical problem on modern 
computer systems . 



,1-0 programming system, with any single or multiprocessor computers, 
j Example multithreaded programming systems include Windows NT, 
M UNIX/Pthreads and Java. 

rfl Other examples than those discussed above can of course be 

a used. While the three examples discussed above are 

ji-5 computationally intensive, other computationally intensive 

**Z systems include volume rendering, terrain masking, threat 

analysis, protein folding, and molecular dynamics simulation. 

As can be seen from the above, the system of the present 
application is highly advantageous and produces significant 
2 0 advantages. 

Although only a few embodiments have been disclosed in 
detail above, other modifications are possible, and would 
understood by those having ordinary skill in the art reading the 
application. For example, although this application has only 



The present application can be used in multithreaded 
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described certain operating systems which capable of handling 
multiple threads, it should be understood that other operating 
systems could be provided. A non-exhaustive list of operating 
systems includes Windows NT, Windows 2000, Java, UNIX, Linux or 
5 any other type system. 

All such modifications are intended to be encompassed 
within the following claims, in which: 

1 
2 



- 57 - 



